Many machine-learning classification systems depend on machine-learning classifiers to detect when unknown samples come from a population of interest (e.g., spam, targeted emails, or malware). Typically, these classifiers are trained using a set of training data that includes samples that are known or believed to come from the population of interest.
Unfortunately, traditional machine-learning approaches to classification may fall short since a traditional machine-learning classifier may not perform well when asked to classify an unknown sample that arises from a different distribution than that of the classifier's training set and will likely return a prediction that is not correct or useful when classifying such samples. For example, a machine-learning classifier that is trained using samples taken, during a particular period of time, from a population whose underlying distribution changes over time may not be able to accurately identify samples taken from the population during a subsequent period of time. Likewise, a machine-learning classifier that was trained using samples taken from a population whose distribution is not representative of the underlying distribution of the population or a certain subpopulation may not be able to accurately identify all samples taken from the population. For example, a classifier that is trained to recognize malware using a training dataset containing only obfuscated malware may not produce any sensible prediction when given non-obfuscated malware to classify. Similarly, a classifier that is trained to recognize sentences as “offensive” or “not offensive” using a training dataset containing only English words may not produce any sensible prediction when given a German sentence to classify. The instant disclosure, therefore, identifies and addresses a need for systems and methods for detecting low-density training regions of machine-learning classification systems.